The present invention relates to an image processing technique for synthesizing images, taken by multiple cameras, through various types of processing including deformation and integration. More particularly, the present invention relates to a technique effectively applicable to a monitoring system as an aid for vehicle driving operations, for example.
Recently, car-mounted display devices have been more and more popularized and the prices of video capturing devices, including video cameras, have been reduced drastically. Reflecting such tendencies, a system usable as a safety check aid for a vehicle driver by allowing him or her to monitor the vehicle's surroundings using video cameras is now available at a reasonable price and is being popularized now.
A system as disclosed in Japanese Patent Application No. 10-217261 is an exemplary vehicle surrounding monitoring system of that type. Specifically, the system combines together images that have been taken by multiple cameras mounted on a vehicle to produce a synthesized image, which looks as if the vehicle were seen from a virtual viewpoint located over the vehicle, and then presents the virtual image to the driver.
FIG. 19 illustrates an exemplary configuration for a system of that type. As shown in FIG. 19, the system includes: camera section 40 made up of multiple cameras 401; image processing section 50 coupled to the camera section 40 via transmission lines 45; and display device 60. The image processing section 50 includes: buffer memories 501 associated with the respective cameras 401; mapping table 502 describing a relationship between a synthesized image that should look as such to a virtual viewpoint and the images actually taken by the cameras 401; and image synthesizer 503. Each of the cameras 401 included in the camera section 40 outputs an image signal on a field-by-field or frame-by-frame basis. The image signal output is transmitted through associated one of the transmission lines 45 and then stored on associated one of the buffer memories 501. By reference to the data stored on the mapping table 502, the image synthesizer 503 combines together the images signals, read out from the buffer memories 501, to produce a synthesized image and present it on the display device 60. By looking at the synthesized image produced by the system shown in FIG. 19, the user (i.e., the driver of a vehicle) can know an exact positional relationship between his or her own vehicle and the vehicle's surroundings without taking the actual positions of the cameras into consideration. Accordingly, this system allows the driver to park his or her vehicle much more safely, for example.
The system of this type, however, has the following drawbacks.
As shown in FIG. 19, each of the transmission lines 45 is associated with one of the cameras 401. That is to say, the same number of transmission lines 45 as that of the cameras 401 should connect the cameras 401 to the image processing section 50. Normally, the cameras 401 are mounted at various positions of a vehicle to produce a synthesized image representing the vehicle's surroundings. Accordingly, the vehicle should be wired with the transmission lines 45 here and there. As a result, it takes too much time and trouble to install this system on the vehicle. Also, such a system requires a great deal of maintenance to avoid failures, for example.
Stated otherwise, to make the system more easily installable and maintainable, the number of transmission lines should be reduced by getting one transmission line shared by multiple cameras. However, it usually takes a huge channel capacity to transfer the image data of each camera entirely. For that reason, it would be hard for a reduced number of transmission lines to cope with such a tall demand. That is to say, to reduce the number of transmission lines, there is no other choice than cutting down the amount of image data to be transferred.
Also, the image processing section 50 needs to store that huge amount of image data that has been transferred from the cameras 401 on a field-by-field or frame-by-frame basis. Thus, each of the buffer memories 501 should have a great storage capacity.
Furthermore, although a synthesized image is produced from a plurality of images taken by multiple cameras, not all of each camera image is used for the image synthesis but each image contains some unnecessary parts. Moreover, even in an image area required for the image synthesis, some part of the area should show a resolution different from that of another during the image synthesis process. Accordingly, it is not always necessary to transmit the entire image data of each camera image as it is to the image processing section.